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TITLE OF THE INVENTION 

INFORMATION PROCESSING METHOD, APPARATUS, PROGRAM AND 
STORAGE MEDIUM STORING SAID PROGRAM 

5 

FIELD OF THE INVENTION 
This invention relates to a technique for 
creating (developing) standard- format image data that 
includes a description of reversibly compressed or 
10 non-compressed digital image data (raw data) obtained 
by digitally converting a signal that is output from 
an image sensing device. 

BACKGROUND OF THE INVENTION 
15 In general, image data that has been obtained by 

an image sensor having an image sensing device such as 
a CCD or CMOS is subjected to image processing within 
the image sensor and is converted (developed) into 
reversibly compressed or non-compressed standard- 
20 format image data such as image data in the TIFF 

format. The reason for adopting reversibly compressed 
or non-compressed image data is to put the image data 
in a form in which the image data that prevailed at 
the time of capture will not be lost. For example, as 
25 indicated by operation 1 in an image sensor 101 shown 
in Fig. 14, the general practice is for the image 
sensor itself to execute all of a series of processing 
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operations that include converting (image processing) 
a signal obtained from an image sensing device such as 
a CCD to standard image data reproducible by another 
device and writing this image data to a recording 
5 medium. However, in a system of this kind, a user 
cannot reproduce the image data upon changing image 
processing parameters such as white balance and gamma 
conversion after photography. 

An expedient for dealing with the above problem 

10 is to make it possible for an external device to be 

supplied with the digital image data (raw data) per se 
of a CCD signal or the like obtained by an imaging 
operation. In accordance with this approach, a user 
is capable of employing a personal computer or the 

15 like to create image data upon changing image 

processing parameters such as white balance and gamma 
conversion of image data that has been obtained by an 
image sensor. According to this arrangement, there is 
an implementation, as indicated by operation 2 in the 

20 image sensor 101 of Fig. 14, in which digital image 

data obtained as a result of imaging is recorded on a 
recording medium as is, the recording medium is loaded 
in a personal computer 102 and the digital image data 
is read out to the personal computer 102 from the 

25 recording medium, or the image data is transmitted to 
the personal computer 102 directly from the image 
sensor by communication. As a result, the digital 
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image data is subjected to processing (processing 
corresponding to operation 1) similar to that within 
the image sensor by software running on the personal 
computer 102, whereby the image data is converted to 
5 standard image data and developed. 

However, image development processing, etc., 
performed by a product A that uses a CCD having a 
primary-color filter (a primary-color CCD) differs 
from that performed by a product B that uses a CCD 

10 having a complimentary- color filter (a complementary- 
color CCD). As a consequence, the prior art is such 
that with the arrangement for implementing operation 2 
described above, signal processing software for 
executing a plurality of types of processing for 

15 generating a luminance signal and/or processing for 

generating a color signal must be selected by the user 
manually in accordance with each product. In addition, 
in a case where image data is transferred to a 
personal computer or the like and the image data is to 

20 be processed by the personal computer, the user does 

not know what signal processing method to apply to the 
raw data that has been output from the image sensing 
device, i.e., whether signal processing for a primary- 
color filter or signal processing for a complementary - 

25 color filter should be executed. Hence, there is the 
possibility that the wrong signal processing will be 
executed. 
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SUMMARY OF THE INVENTION 
Accordingly, an object of the present invention 
is to make it possible to apply the appropriate image 
5 processing automatically to data that includes 

reversibly compressed image data or non- compressed 
image data obtained by digitally converting a signal 
output from an image sensing device. 

According to the present invention, the foregoing 

10 object is attained by providing an information 

processing method for processing a file containing 
reversibly compressed or non -compressed digital image 
data obtained by digitally converting a signal that 
has been output from an image sensing device, the 

15 method including process of: converting the digital 
image data contained in the file to data having a 
prescribed format by selectively executing a signal 
processing of a plurality of types by using any of at 
least a plurality of types of luminance signal 

20 generating processing methods and/or a plurality of 
types of color signal generating processing methods; 
selecting automatically signal processing to be used 
from among the plurality of types of signal processing 
based upon information contained in the file; and 

25 causing the conversion to be executed using the signal 
processing selected in the selecting process so that 
the digital image data contained in the file is 
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converted to data having the prescribed format . 

Other features and advantages of the present 
invention will be apparent from the following 
description taken in conjunction with the accompanying 
5 drawings, in which like reference characters designate 
the same or similar parts throughout the figures 
thereof . 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 The accompanying drawings, which are incorporated 

in and constitute a part of the specification, 
illustrate embodiments of the invention and, together 
with the description, serve to explain the principles 
of the invention. 
15 Fig. 1 is a block diagram illustrating the 

general configuration of an image processing system 
according to a first embodiment of the present 
invention; 

Fig. 2 is a block diagram illustrating an 
20 overview of image processing executed by an image 
processing apparatus according to the first 
embodiment ; 

Fig. 3 is a block diagram illustrating the 
structure of an image processing unit according to the 
25 first embodiment; 

Fig. 4 is a diagram illustrating pixel data of a 
primary-color filter CCD; 
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Fig. 5 is a diagram illustrating pixel data of a 
complementary- color filter CCD; 

Fig. 6 is a diagram illustrating the coefficients 
of a filter operation for creating a luminance signal 
5 from a complementary- color CCD signal; 

Fig. 7 is a diagram illustrating a file structure 
according to the first embodiment; 

Fig. 8 is a diagram illustrating another file 
structure according to the first embodiment; 
10 Fig. 9 is a block diagram illustrating the 

structure of an image processing unit according to a 
second embodiment of the present invention; 

Fig. 10 is a block diagram illustrating the 
structure of an image processing unit according to a 
15 third embodiment of the present invention; 

Fig. 11 is a block diagram illustrating the 
structure of an image processing unit according to a 
fourth embodiment of the present invention; 

Fig. 12 is a diagram illustrating pixel data of a 
20 3 -chip CCD; 

Fig. 13 is a flowchart illustrating the operation 
of an image sensing apparatus and information 
processing apparatus according to the first 
embodiment ; 

25 Fig. 14 is a conceptual view illustrating image 

processing for reproducing CCD data according to the 
prior art; 
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Fig. 15 is a diagram illustrating coefficients in 
a filter operation for creating a luminance signal 
from a primary-color CCD signal; and 

Fig. 16 is a diagram useful in describing 
5 utilization of a table that correlates extensions and 
processing methods. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Preferred embodiments of the present invention 
10 will now be described in detail in accordance with the 
accompanying drawings . 

[ First Embodiment ] 

A system according to a first embodiment of the 
present invention comprises an image sensing apparatus, 

15 which has a CCD sensor employing a CCD as an image 
sensing device, capable of outputting a file that 
includes CCD raw data (referred to as a "CCD signal" 
below), and an information processing apparatus 
capable of processing the CCD signal (file) supplied 

20 by the image sensing apparatus. The image sensing 

device may be a CMOS sensor or a sensing device of the 
type that separates color by a photoelectric 
transducer having a multilayer structure. 

Fig. 1 is a block diagram illustrating the 

25 general configuration of an image processing system 
according to the first embodiment. The system 
includes an image sensing apparatus 11 having an image 
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sensor 111, a file generator 112 , a recording unit 113 
and an output unit 114. The image sensor 111, which 
has a CCD, applies an analog- to-digital conversion to 
an image signal obtained from the CCD and then applies 
5 non- compressing or reversible compressing processing 
to the digital signal, thereby outputting the 
resulting signal as a CCD signal (digital image data) . 
It will suffice if the raw data in this embodiment has 
an image data format in which the output of an image 

10 sensor is recorded so as to be reproducible without 
loss, such as image data at a stage in which the 
digital image data obtained by the A/D conversion has 
not been subjected to image processing such as at 
least white balance processing, data at a stage in 

15 which the A/D-converted digital image data obtained 
from the image sensing device has not been subjected 
to color- separation processing for separation into a 
luminance signal and color signal, or data at a stage 
in which the output signal of a color filter has not 

20 been subjected to color interpolation processing, as 

in a case where a color filter of a Bayer array or the 
like is employed. 

This embodiment assumes use of a complementary- 
color CCD or primary-color CCD. The file generator 

25 112 generates a file for describing the CCD that has 

been output. The file generator 112 generates a file, 
which is shown in Fig. 7 or Fig. 8, upon adding on tag 
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information that describes attribute information of 
the image sensing apparatus, such as the manufacturer 
name or product name, for specifying the image sensing 
apparatus. The details of the file structure will be 
5 described later with reference to Fig. "7 or Fig. 8. 
It should be noted that since the tag information is 
appended individually to each image obtained by- 
sensing, the tag information may contain information 
relating to the imaging conditions, such as whether or 

10 not a flash was fired. 

The recording unit 113 records the file generated 
by the file generator 112 on a prescribed medium. A 
floppy disk (registered trademark) , magneto -optic disc, 
mini disc, compact disc, compact flash (registered 

15 trademark) or smart media (registered trademark), etc., 
may be used as the medium. The output unit 114 has, 
e.g., a USB interface and is capable of outputting a 
file to an external device. 

An information processing apparatus 12 includes a 

20 CPU 121, a ROM 122, a RAM 123, an interface 124, an 
input panel 125, a display unit 126, an external 
storage device 127, a media drive 128 and a bus 129 
for connecting these components . 

The CPU 121 executes a control program, which has 

25 been stored in the ROM 122 or RAM 123, thereby 
executing various processing inclusive of image 
processing described later. The interface 124 is an 
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interface for performing communication with an 
external device. A USB interface, for example, may be 
used as the interface 124. In this embodiment, a file 
is received from the image sensing apparatus .11 via 
5 the interface 124. The input panel 125, which 
includes a pointing device and a keyboard, etc., 
accepts operating commands from the user. The display 
unit 126 presents various displays under the control 
of the CPU 121. The external storage device 127 

10 stores various application programs and data. In 
general, a hard disk is employed as the external 
storage device 127. The media drive 128 controls data 
communication between a prescribed recording medium 
and the bus 129. 

15 Fig. 13 is a flowchart for describing an overview 

of image processing executed by the information 
processing apparatus 12 using the signal processing 
method according to this embodiment. Steps S101 to 
S103 in Fig. 13 illustrate the operation of the image 

20 sensing apparatus 11, and step S201 to S204 show the 
operation of the information processing apparatus 12. 
Specifically, imaging is performed by the image sensor 
111 to acquire a CCD signal at step S101. Next, at 
step S102, the file generator 112 generates data that 

25 describes the CCD signal acquired from the image 

sensor 111 and adds tag information (described later) 
onto this data to thereby generate a file. This is 
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followed by step S103, at which the file generated at 
step S102 is supplied to the recording unit 113 and 
recorded on a prescribed recording medium. 
Alternatively, in response to a request from an 
5 external device (the information processing apparatus 
12 in this embodiment), the output unit 114 outputs a 
file that has been recorded on the recording unit 113. 

Upon accepting a file thus generated, the 
information processing apparatus 12 stores the file in 

10 the external storage device 127 or RAM 123 temporarily. 
The control program for implementing the image 
processing of this embodiment has been stored in the 
external storage device. The program in loaded into 
the RAM 123 and executed by the CPU 121 as necessary. 

15 When image processing according to this 

embodiment is executed, a file to be processed is read 
into the RAM 123 from the external storage device 127 
at step S201. The file may be read in from the media 
drive 128 or from the image sensing apparatus 11 via 

20 the interface 124. Next, at step S202, this file is 

analyzed and the color composition or array of the CCD 
signal (whether the CCD signal is a primary-color CCD 
signal or complementary- color CCD signal according to 
this embodiment) is discriminated. This is followed 

25 by step S203, at which appropriate image processing is 
selected in accordance with the color composition or 
array of the CCD signal discriminated at step S202, 
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the CCD signal is processed and undergoes a conversion 
to the format of ordinary image data. The data 
resulting from the format conversion is stored on a 
recording medium (the external storage device 127 or 
5 media drive 128) at step S204. 

Fig. 2 is a block diagram useful in describing a 
functional arrangement for implementing the foregoing 
image processing (steps S202 to S204) in the 
information processing apparatus 12 of the first 

10 embodiment. The flow of data in image processing 

according to this embodiment will be described with 
reference to Fig. 2. 

File data that describes a CCD signal is analyzed 
by a format analyzer 201. More specifically, 

15 information indicating the number of pixels of the CCD 
that generated this CCD signal, photographic 
conditions and product information (product name 
contained in the tag information) for identifying the 
product are read in by the format analyzer 201. 

20 Fig. 7 is a diagram illustrating an example of 

the data structure of a file used in this embodiment. 
A product name contained in tag information 702 is 
read out as product information by the format analyzer 
201. The product name that has been read out is 

25 compared by the format analyzer 201 with a product 
list in which product names and characteristics of 
cameras having these product names are arranged in 
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corresponding form, whereby it is determined whether 
the product is a camera that uses a primary-color CCD 
or a camera that uses a complementary- color CCD. 
Though the product name is adopted as the product 
5 information is this embodiment, a combination of 

product name and manufacturer name may also be adopted 
as the product information. It should be noted that 
the product list is stored in the external storage 
device 127, etc., beforehand and is loaded into the 

10 RAM 123 and referred to as necessary. The CCD signal 
is described in image data 703. 

A CCD signal reader 202 reads the CCD signal from 
the file data on the basis of the information that has 
been read out by the format analyzer 201. The CCD 

15 signal reader 202 is capable of executing various 

types of decompression processing. By way of example, 
the CCD signal reader 202 reads compression format 
information from the tag information in the file 
obtained by the format analyzer 201, decides the 

20 decompression format in accordance with the 

compression format information, reads out the CCD 
signal and then decompresses the signal. 

The result of the determination made by the 
format analyzer 201 based upon the product information 

25 as to whether the camera is a camera using a primary- 
color CCD or a camera using a complementary- color CCD 
indicates whether the CCD signal that has been read by 
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the CCD signal reader 202 is a primary- color CCD 
signal or a complementary- color CCD signal. On the 
basis of the result of the determination made by the 
format analyzer 201, a processing method decision unit 
5 203 decides the image processing to be used in an 
image processor 204, which is the next stage, and 
supplies this to the image processor 204 as processing 
selection information. The processing method decision 
unit 203, which holds a table or the like in which 

10 results of determinations and image processing methods 
are arranged in corresponding form, outputs the 
processing selection information by referring to this 
table. As a result, image processing can be changed 
over based upon the type of CCD signal (in this 

15 embodiment, whether the signal is a primary-color CCD 
signal or a complementary- color CCD signal). 

The CCD signal that has been read by the CCD 
signal reader 202 is sent to the image processor 204. 
On the basis of a parameter set by a parameter setting 

20 unit 207, the image processor 204 subjects the CCD 

signal to the processing that has been selected by the 
processing method decision unit 203. The parameter 
setting unit 207 provides the image processor 204 with 
a parameter that conforms to the processing method 

25 decided. Further, since the setting of the parameter 
in the parameter setting unit 207 can be changed 
depending upon user preference, it is possible to set 
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a parameter such as white balance, color density, hue, 
intensity of edge emphasis and contrast. For example, 
in a change of parameter by the user, the parameter 
that has been selected in accordance with the 
5 processing method is adopted as the default value and 
an increase or decrease can be applied based upon a 
value set by the user. The output image signal that 
has been processed by the image processor 204 is 
subjected to a format conversion by a format converter 
10 205 so as to be converted to the JPEG format, TIFF 

format or BMP format, etc., and the resulting signal 
is written to the recording medium by an output unit 
206. 

The details of the image processor 204 will be 
15 described next. Fig. 3 is a block diagram useful in 
describing the details of processing included in 
processing executed by the image processor 204. The 
flow of image processing in the image processor 204 
will be described with reference to the block diagram 
20 of Fig. 3. 

The CCD signal that has entered from the CCD 
signal reader 202 is sent first to a white balance 
processor 301, where white balance coefficients and 
color temperature of the light source that will cause 
25 white in the image to become a white signal are 

obtained. White balance processing is executed in 
such a manner that white in the image that is the 
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result of applying the white balance coefficients to 
the CCD signal will become a white signal. The CCD 
signal that has been subjected to the white balance 
processing is sent to a U, V signal (color difference 
5 signal) generating channel (also referred to as a 

"color signal generating channel" , which is composed 
of units from an interpolating processor 302 to a 
color difference signal converting processor 307) and 
to a Y signal (luminance signal) generating channel 

10 (composed of units from a luminance signal creation 
selector 308 to a gamma processor 312). 

The U, V signal generating channel will be 
described first. Using pixel values at positions A, B # 
C, D from the pixel array of a single-chip CCD, the 

15 interpolating processor 302 generates field data of A, 
B, C, D signals by an interpolating operation. For 
example, we have (A,B,C,D) = (R,G1,G2,B) in case of a 
primary-color CCD signal in Fig. 4, and we have 
(A,B,C,D) = (C,M,Y,G) in case of a complementary- color 

20 CCD signal in Fig. 5. A matrix calculation unit 303 
performs a color conversion pixel by pixel using 
Equation (1) below. It should be noted that the 
matrix elements Mil to M43 in Equation (1) are changed 
and set by the parameter setting unit 207 in 

25 dependence upon the processing method (processing for 
a primary-color CCD signal or processing for a 
complementary- color CCD signal in this embodiment) 
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decided by the processing method decision unit 203. 
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(1) 



The CCD signal that has undergone the color 
conversion to Rm, Gm, Bm by the matrix processor 303 
5 is input to a color-difference gain processor 304. 
The latter acquires a luminance signal Y and color 
difference signals Cr, Cb from the entered CCD signal, 
multiplies the color difference signals by a gain to 
generate Cr' , Cb and converts Y, Cr 1 , Cb ' again to 

10 signals in RGB color space. More specifically, the 

color-difference gain processor 304 first converts the 
Rm, Gm, Bm signals to Y, Cr, Cb signals by Equation 
(2) below, multiplies the Cr, Cb signals by gain in 
accordance with Equation (3) below, and converts the Y, 

15 Cr' , Cb 1 obtained to Rg, Gg, Bg signals according to 

Equation (4) [which is a matrix that is the inverse of 
the matrix of Equation (2)]. It should be noted that 
the gain coefficient Gl in Equation (3) is changed and 
set by the parameter setting unit 207 in dependence 

20 upon the processing method decided by the processing 
method decision unit 203, namely processing for the 
primary-color CCD signal or processing for the 
complementary-color signal. 
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The CCD signal obtained by the color-difference 
5 gain processor 304 is sent to a gamma processor 305. 
The latter converts the entered CCD signal to data 
using the following Equations (5) to (7): 

Rt = GammaTable [Rg] ... 

(5) 

10 Gt = GammaTable [Gg] ... 

(6) 

Bt = GammaTable [Bg] ... 

(7) 

where GammaTable in the above equations represents a 
15 one -dimensional look-up table (i.e. # a gamma table). 
The gamma table is changed and set by the parameter 
setting unit 207 in dependence upon the processing 
method decided by the processing method decision unit 
203. 

20 Next, the CCD signal that has undergone the gamma 

correction is sent to a hue correcting processor 306. 
The latter first converts the Rt, Gt, Bt signals 
obtained by the gamma processor 305 to Y, Cr, Cb 
signals by Equation (8) below and then subjects Cr, Cb 

25 to a signal correction by Equation (9) below and 
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10 



15 



20 



effects a conversion to Rt, Gt, Bt signals according 

to Equation (10) [which is a matrix that is the 

inverse of the matrix of Equation (9)]. The matrix 

elements Hll to H22 in Equation (9) (where H signifies 

the hue) are changed and set by the parameter setting 

unit 207 in dependence upon the processing method 

(processing for a primary- color CCD signal or 

processing for a complementary- color CCD signal in 

this embodiment) decided by the processing method 

decision unit 203. 
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(10) 



The CCD signal (Rt,Gt,Bt) that has been output 

from the hue correcting processor 306 is sent to the 

color difference signal converting processor 307. The 

latter creates U, V signals from Rt, Gt, Bt using 

Equation (11) below. The color difference signals U, 

V are generated in the manner described above . 
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Meanwhile, the processing set forth below is 
executed in the channel for generating the luminance 
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signal. The CCD signal that has undergone white 
balance processing is input to a primary- color 
luminance signal creating processor 309 or 
complementary-color luminance signal creating 
5 processor 310 by the luminance signal creation 

selector 308. That is, the luminance signal creation 
selector 308 decides the destination of the CCD signal 
in accordance with the processing selection 
information that enters from the processing method 

10 decision unit 203. As a result, the luminance signal 
is created by the primary-color luminance signal 
creating processor 309 in case of the primary-color 
CCD signal and the luminance signal is created by the 
complementary- color luminance signal creating 

15 processor 310 in case of the complementary- color CCD 
signal. 

More specifically, the primary- color CCD signal 
of the kind shown in Fig. 4 is supplied to the 
primary-color luminance signal creating processor 309. 

20 The latter makes the R, B signals in the CCD all "0"s, 
applies two-dimensional filter processing having the 
coefficients shown in Fig. 15 and adopts the resulting 
signal as the luminance signal. Fig. 15 shows an 
interpolating filter for interpolating the G signal. 

25 On the other hand, the complementary- color CCD signal 
of the kind shown in Fig. 5 is supplied to the 
complementary- color luminance signal creating 
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processor 310. The latter applies two-dimensional 
filter processing having the coefficients shown in Fig. 
6 to the CCD signal (without subjecting the CCD signal 
to any other processing) and adopts the resulting 
5 signal as the luminance signal. Fig. 6 illustrates 
one type of low-pass filter. 

The luminance signal created by the primary- color 
luminance signal creating processor 309 or 
complementary- color luminance signal creating 

10 processor 310 is subjected to edge emphasizing 

processing by a high-frequency emphasizing processor 
311. Gamma correction processing is applied by the 
gamma processor 312 to thereby create the final Y 
signal. The gamma processor 312 is a gamma processor 

15 for the luminance signal. 

It should be noted that parameters that can be 
changed by a user setting employing the parameter 
setting unit 207 are Gl in Equation (3), the table 
data in Equations (5) to (7) and, though these have 

20 not been described in this embodiment, the intensity 
of edge emphasis by the high-frequency emphasizing 
processor 311 and a table, etc., used in the gamma 
processor 312. The parameter that has been set by the 
user is stored in an internal memory. (not shown) of 

25 the information processing apparatus or in an external 
storage device or recording medium, etc., together 
with the CCD signal. The parameter may be adopted as 
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an initial value at the time of signal creation and 
can be used upon being read out when appropriate. 
Further, the parameter changed over depending upon the 
complementary- color CCD signal and primary- color CCD 
5 signal is one obtained by applying a user setting to a 
default value corresponding to the primary-color 
filter or complementary-color filter (i.e., a value 
set by the user is added to or subtracted from the 
default value), as indicated by Gl in Equation (3) or 

10 Hll to H22 in Equation (9). It should be noted that 
the input of the value set by the user is made by the 
input panel 125 and a user interface presented by the 
display unit 126. 

The foregoing is the flow of image processing 

15 according to the first embodiment. It should be noted 
that although the parameter setting unit 207 of Fig. 3 
changes parameters in dependence upon the processing 
method decided by the processing method decision unit 
203, this does not impose a limitation upon the 

20 invention. For example, it may be so arranged that 

parameters are decided based upon product information 
acquired by the format analyzer 201. 

Further, in this embodiment, tag information in a 
file is analyzed by the format analyzer 201 of Fig. 2 

25 and selection of the processing method in the image 
processor 204 is performed based upon the product 
information obtained. However, as shown in Fig. 8, it 
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may be so arranged that in a case where CCD filter 
information indicating color information and color 
array of the CCD signal (color filter) is contained in 
tag information 802 of the file, the processing method 
5 of the image processor 204 is changed over based upon 
this CCD filter information [changeover of the color 
signal generating channel (based upon the parameter) 
and changeover of the processing unit in the luminance 
signal generating channel] . Further, though an 

10 arrangement having the primary-color luminance signal 
creating processor 309 and complementary- color 
luminance signal creating processor 310 has been 
illustrated as the luminance signal creating processor, 
this may be combined with a luminance signal creating 

15 processor that processes another type of CCD signal or 
the arrangement may have three or more types of 
luminance signal creating processors. For example, it 
should be obvious from the description of the above 
embodiment that a plurality of types of signal 

20 processing may be provided, such as luminance signal 
creation processing supporting a primary-color filter 
and complementary- color filter of different arrays, a 
luminance signal creating processor supporting a 3- 
chip CCD signal described in a fourth embodiment, and 

25 a luminance signal creating processor supporting the 
output signal of an image sensing device of the type 
that separates color by a photoelectric transducer 
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having a multilayer structure. It is of course 
possible to process these CCD signals by changing over 
parameters in the color signal generating channel. 

Thus, in accordance with the first embodiment, as 
5 described above, there is provided a signal processing 
method for reading a CCD signal out of a file in which 
an output signal from a CCD is described in a camera- 
specific signal format, and converting this CCD signal 
to a standard signal. The method includes 

10 automatically selecting one type of luminance signal 
creation processing from at least two types of 
luminance signal creation processing prepared in 
advance by referring to tag information in the file, 
and effecting a conversion from the CCD signal to the 

15 luminance signal using a luminance signal that has 
been created by the luminance signal creation 
processing selected. Here the type of CCD signal can 
be discriminated by referring to a product name or CCD 
filter information contained in the tag information. 

20 As a result, the appropriate image processing is 

applied automatically in accordance with the type of 
CCD signal (e.g., the type of colors or array in the 
CCD signal, such as a complementary-color CCD signal 
or primary-color CCD signal) and the conversion to the 

25 standard signal is performed correctly. 
[Second Embodiment] 

A second embodiment of the present invention will 
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be described with reference to Fig. 9. The 
configuration of the system and the essentials of 
processing in the image sensing apparatus 11 and 
information processing apparatus 12 of the second 
5 embodiment are similar to those of the first 

embodiment (Figs. 1, 2 and 13). The structure and 
operation of the image processor 204 according to the 
. second embodiment will now be described with reference 
to Fig. 9. 

10 Fig. 9 is a block diagram useful in describing 

the structure of the image processor 204 according to 
the second embodiment. In Fig. 9, the structure of a 
white balance processor 901 and the structure of the U, 
V signal generating channel (from an interpolating 

15 processor 902 to a color difference signal converting 
processor 907) are similar to those of the first 
embodiment (from the interpolating processor 302 to 
the color difference signal converting processor 307). 
Processing for generating the Y signal according 

20 to the second embodiment will be described. In the 
first embodiment, the luminance signal creating 
processor in the luminance signal generating channel 
is changed over in dependence upon the type of CCD 
signal. In the second embodiment, however, high- 

25 frequency emphasizing processing for edge emphasis in 
the luminance signal generating channel is changed 
over in dependence upon the type of CCD signal. 
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In Fig. 9, the CCD signal that has undergone 
white balance processing is sent to a luminance signal 
creating processor 908, which proceeds to execute 
processing for lowering the frequency band using the 
5 filter coefficients of Fig. 6. The luminance signal 
thus created is sent to a high-frequency signal 
strength adjusting processor 912, which is the next 
stage. Next, in accordance with the processing 
selection information from the processing method 

10 decision unit 203, a high-frequency signal creation 
selector 909 selects a primary-color high-frequency 
signal creating processor 910 or a complementary- color 
high-frequency signal creating processor 911 as the 
destination for supply of the CCD signal that enters 

15 from the luminance signal creating processor 908. 

That is, the processing method for creating a high- 
frequency signal with regard to the CCD signal is 
selected by the processing selection information 
decided based upon the product information. As a 

20 result, the high-frequency signal is created using the 
primary-color high-frequency signal creating processor 
910 in the case of the primary-color CCD signal (Fig. 
4) and using the complementary-color high-frequency 
signal creating processor 911 in the case of the 

25 complementary- color CCD signal (Fig. 5). 

The primary- color high-frequency signal creating 
processor 910 reads in the CCD signal that has been 
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subjected to white balance processing by the white 
balance processor 901, makes the values at the pixel 
positions of the R, B signals "0", executes filter 
processing using the filter shown in Fig. 15 based 
5 upon the Gl , G2 signals and creates a G interpolation 
signal. The primary-color high-frequency signal 
creating processor 910 further subjects the G 
interpolation signal to [-1, 0, 2, 0, -1] filter 
processing in the horizontal direction and [-1, 0, 2, 

10 0, -1] filter processing in the vertical direction, 
thereby creating a high-frequency signal. 

Further, the complementary- color high-frequency 
signal creating processor 911 subjects the luminance 
signal, which has been sent from the luminance signal 

15 creating processor 908, to [-1, 0, 2, 0, -1] filter 

processing in the horizontal direction and [-1, 0, 2, 
0, -1] filter processing in the vertical direction, 
thereby creating a high-frequency signal. 

The high-frequency signal created by the primary - 

20 color high-frequency signal creating processor 910 or 
complementary-color high-frequency signal creating 
processor 911 has its intensity adjusted by a gain 
operation in the high-frequency signal strength 
adjusting processor 912, which then proceeds to add 

25 the resultant signal to the luminance signal created 
by the luminance, signal creating processor 908. The 
luminance signal to which the high-frequency signal 
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has been added undergoes a gamma conversion in a gamma 
processor 913, whereby the Y signal is created. 

Thus, in accordance with the method of the second 
embodiment, the method includes automatically 
5 selecting one type of high-frequency emphasis signal 
creation processing from at least two types of high- 
frequency emphasis signal creation processing prepared 
in advance, and generating a luminance signal that has 
been subjected to the high-frequency emphasis signal 

10 creation processing selected. As a result, the 

appropriate image processing is applied automatically 
in accordance with the type of CCD signal and the 
conversion to the standard signal is performed 
correctly. 

15 [Third Embodiment] 

In the first embodiment, signal processing in a 
luminance signal creating processor differs depending 
upon whether the CCD signal is a primary-color CCD 
signal or a complementary- color CCD signal. In the 

20 second embodiment, signal processing in a high- 
frequency emphasis signal creating processor differs 
depending upon whether the CCD signal is a primary- 
color CCD signal or a complementary- color CCD signal. 
In a third embodiment, both luminance signal creation 

25 and high-frequency emphasizing processing are changed 
over depending upon whether the CCD signal is a 
primary- color CCD signal or a complementary- color CCD 
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signal. 

Fig. 10 is a block diagram useful in describing 
the structure of the image processor 204 according to 
the second embodiment. In Fig. 10, the. structure of a 
5 white balance processor 1001 and the structure of the 
U, V signal generating channel (from an interpolating 
processor 1002 to a color difference signal converting 
processor 1007) are similar to those of the first 
embodiment (from the interpolating processor 302 to 

10 the color difference signal converting processor 307). 
Further, a luminance signal creation selector 
1008, primary-color luminance signal creating 
processor 1009 and complementary- color luminance 
signal creating processor 1011 have functions 

15 identical with those of the luminance signal creation 
selector 308, primary-color luminance signal creating 
processor 309 and complementary- color luminance signal 
creating processor 310, respectively, of the first 
embodiment . 

20 The luminance signal generated by the primary- 

color luminance signal creating processor 1009 
undergoes high-frequency emphasis in a primary- color 
high-frequency emphasizing processor 1010. On the 
other hand, the luminance signal generated by the 

25 complementary- color luminance signal creating 

processor 1011 undergoes high-frequency emphasis in a. 
complementary- color high-frequency emphasizing 
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processor 1012. It should be noted that the primary- 
color high-frequency emphasizing processor 1010 has 
functions identical with those of an arrangement 
obtained by combining the primary- color high-frequency 
5 signal creating processor 910 and high-frequency 

signal strength adjusting processor 912 of the second 
embodiment. Similarly, the complementary-color high- 
frequency emphasizing processor 1012 has functions 
identical with those of an arrangement obtained by 
10 combining the complementary -color high-frequency 
signal creating processor 911 and high-frequency 
signal strength adjusting processor 912 of the second 
embodiment . 

The luminance signal whose high frequency has 
15 thus been emphasized is subjected to a gamma 

conversion in a gamma processor 1013 , whereby the Y 
signal is created. 

[Fourth Embodiment] 

In a fourth embodiment of the present invention, 
20 it is possible to appropriately process a CCD signal 
obtained via a primary-color filter of a single-chip 
Bayer array of the kind shown in Fig. 4 (this signal 
shall be referred to as a "Bayer CCD signal" in this 
embodiment) and a CCD signal obtained by a 3-chip CCD 
25 of the kind shown in Fig. 12 (this signal shall be 
referred to as a "3-chip CCD signal" in this 
embodiment ) . 



- 30 - 



CFM03504/P204-0064 



The configuration of the system and the 
essentials of processing in the image sensing 
apparatus 11 and information processing apparatus 12 
of the fourth embodiment are similar to those of the 
5 first embodiment (Figs. 1 # 2 and 13). The structure 
and operation of the image processor 204 according to 
the fourth embodiment will now be described with 
reference to Fig. 11. 

When file data is supplied to the format analyzer 

10 201, the latter analyzes the tag information contained 
in the file. More specifically, information 
indicating the number of pixels of the CCD, 
photographic conditions and product information for 
identifying the product are read in by the format 

15 analyzer 201. The CCD signal is read out by the CCD 
signal reader 202 based upon the information read out 
by the format analyzer 201. Next, on the basis of 
file array information of the CCD read out by the 
format analyzer 201, the processing method decision 

20 unit 203 determines whether the CCD signal read out by 
the CCD signal reader 202 is the Bayer CCD signal or 
the 3 -chip CCD signal and generates processing 
selection information for deciding the processing 
executed in the image processor 204. 

25 Fig. 11 is a block diagram useful in describing 

the structure of the image processor 204 according to 
the fourth embodiment. In Fig. 11, the structure of a 
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white balance processor 1101 and the structure of the 
U, V signal generating channel (an interpolating 
processor 1102 to a color difference signal converting 
processor 1107) are similar to those of the first 
5 embodiment (the interpolating processor 302 to the 
color difference signal converting processor 307). 

The CCD signal that has undergone white balance 
processing is supplied to either a single-chip Bayer 
luminance signal creating processor 1109 or 3-chip CCD 

10 luminance signal creating processor 1110 by a 

luminance signal creation selector 1108 based upon the 
processing selection information. The 3-chip CCD 
luminance signal creating processor 1110 generates a 
luminance signal using color signals of all colors 

15 contained in the CCD signal (digital image data) . 
Further, the single-chip Bayer luminance signal 
creating processor 1109 generates a luminance signal, 
of the kind described in the first to third 
embodiments, using a color signal of a specific color 

20 contained in the CCD signal (digital image data). The 
Bayer CCD signal is sent to the single-chip Bayer 
luminance signal creating processor 1109, which 
proceeds to create a luminance signal. On the other 
hand, the 3-chip CCD signal is sent to the 3-chip CCD 

25 luminance signal creating processor 1110, which 
creates a luminance signal. 

The Bayer CCD signal is identical with the 
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primary-color CCD signal of the first embodiment. 
Accordingly, processing for creating the single-chip 
Bayer luminance signal is the same as processing used 
in the primary- color luminance signal creating 
5 processor of the first embodiment and need not be 
described here. 

The 3 -chip CCD luminance signal creating 
processor 1110 creates the luminance signal of the 3- 
chip CCD signal of the kind shown in Fig. 12 in 
10 accordance with Equation (12) below. 

Y = 0.33 x R + 0.69 x G + 0.11B ... (12) 

The luminance signal created by the single-chip 
Bayer luminance signal creating processor 1109 or 3- 
chip CCD luminance signal creating processor 1110 has 
15 its edge component emphasized by a high-frequency 

emphasizing processor 1111 and is further subjected to 
gamma conversion processing by a gamma processor 1112, 
whereby the final Y signal is generated. 

Thus, in accordance with the fourth embodiment, 
20 processing for creating a luminance signal can be 

changed over upon distinguishing between a single -chip 
primary- color CCD signal and a 3 -chip primary-color 
CCD signal. It should be noted that the changeover in 
processing for creating the luminance signal in the 
25 fourth embodiment may be achieved in combination with 
the first or third embodiment. Furthermore, in a case 
where the invention is applied to the third embodiment. 
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it is assumed that the 3 -chip CCD signal and primary- 
color CCD signal (Bayer CCD signal) are both processed 
by the primary-color high-frequency emphasizing 
processor 1010. It is of course possible to combine 
5 the complementary- color CCD signal as the single-chip 
CCD signal. For example, in a case where the 
arrangement of the third embodiment is employed in 
this combination, the 3 -chip CCD signal is processed 
by the primary-color high-frequency emphasizing 
10 processor 1010 and the complementary-color CCD signal 
is processed by the complementary- color luminance 
signal creating processor 1011. 
[Fifth Embodiment] 

In the first embodiment, information contained in 
15 a file is analyzed by the format analyzer 201. More 
specifically, information indicating the number of 
pixels of the CCD, photographic conditions and product 
information for identifying the product are read in by 
the format analyzer 201. The CCD signal is read out 
20 by the CCD signal reader 202 based upon the 

information read out by the format analyzer 201. In a 
fifth embodiment, a file extension is read out by the 
format analyzer 201 and, in accordance with the 
extension, the CCD signal reader 202 determines the 
25 processing for generating the luminance signal in the 
image processor 204 (luminance signal processing 
conforming to the primary- color CCD signal or 
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complementary- color CCD signal) and the processing for 
decompressing a compressed file. 

An arrangement of the kind shown in Fig. 16 is 
furnished in order to change over processing 
5 automatically in accordance with the extension of the 
file data. Specifically, it will suffice if signal 
processing (decompression method and processing 
method) corresponding to each of a plurality of 
extensions of file data are retained in the form of a 

10 table 210. The CCD signal reader 202 acquires data 

indicating the extension from the format analyzer 201, 
refers to the table 210 and then selects the 
decompression method. Similarly, the processing 
method decision unit 203 acquires data indicating the 

15 extension from the format analyzer 201, refers to the 
table 210 and then decides the processing method. 
Further, it may be so arranged that an extension list 
(table) storing extensions of files capable of being 
processed is retained and whether entered file data 

20 can be processed or not is discriminated by referring 
to this table. 

It may so arranged that more detailed signal 
processing is discriminated by combining tag 
information along with the file data extensions. 

25 Alternatively, it may be so arranged that in the event 
of input of a file having an extension not contained 
in a table holding the extensions of files capable of 
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being processed, signal processing is changed over 
using tag information possessed by the file data. 
Further, it should be obvious that it is permissible 
to adopt an arrangement in which processing for 
5 decompressing compressed data is decided based upon 
tag information (product name, etc.). 

Further, embodiments in which captured digital 
image data and tag information are integrated have 
been described. However, if conditions are such that 
10 items of digital image data and tag information are 
linked, these may just as well be stored as separate 
files. 

Thus, in accordance with each of the embodiments 
as described above, if image processing is necessary 

15 in order to change the format of data having a 

manufacturer- specif ic format to a standard format, it 
is possible to make the change to the optimum standard 
format automatically without mistaking the image 
processing method. 

20 In accordance with the present invention, as 

described above, even a novice is capable of applying 
appropriate image processing to data that includes 
reversibly compressed or non-compressed digital image 
data obtained by digitally converting a signal that is 

2 5 • output from an image sensing device, and it is 
possible to facilitate operation by the user. 
< Other Embodiment s> 
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It may be so arranged that the functions 
described in the foregoing embodiments are implemented 
in an image sensing apparatus . In such case a 
removable recording medium on which image data 
5 obtained by photography using another camera has been 
recorded is loaded in the apparatus , the image data is 
read out of the recording medium, the processing of 
the above -described embodiments is applied to the 
image data to convert it to image data having a 

10 prescribed format, and the resulting image data is 

recorded on a recording medium or is output externally. 

Further, in the foregoing embodiments, the 
arrangement is such that a known image processing 
method is decided in the processing method decision 

15 unit 203 with respect to product information or the 
like. However, in may be so arranged that in a case 
where product information is so new that there is no 
corresponding image processing method in the reference 
table, the closest image processing method is selected 

20 judging from the version of the product information or 
the name of the manufacturer. 

Further, product information has been described 
with regard to files that can be analyzed. However, 
it may be so arranged that in a case where a format 

25 cannot be read and an extension or file tag 

information per se cannot be analyzed, an error 
display is presented to the user and processing of 
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latter stages is halted automatically so that needless 
processing will not be executed. 

Note that the present invention can be applied to 
an apparatus comprising a single device or to system 
5 constituted by a plurality of devices. 

Furthermore, the invention can be implemented by 
supplying a software program, which implements the 
functions of the foregoing embodiments, directly or 
indirectly to a system or apparatus, reading the 

10 supplied program code with a computer of the system or 
apparatus, and then executing the program code. In 
this case, so long as the system or apparatus has the 
functions of the program, the mode of implementation 
need not rely upon a program. 

15 Accordingly, since the functions of the present 

invention are implemented by computer, the program 
code installed in the computer also implements the 
present invention. In other words, the claims of the 
present invention also cover a computer program for 

20 the purpose of implementing the functions of the 
present invention . 

In this case, so long as the system or apparatus 
has the functions of the program, the program may be 
executed in any form, such as an object code, a 

25 program executed by an interpreter, or scrip data 
supplied to an operating system. 
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Example of storage media that can be used for 
supplying the program are a floppy disk, a hard disk, 
an optical disk, a magneto-optical disk, a CD-ROM, a 
CD-R, a CD-RW, a magnetic tape, a non- volatile type 
5 memory card, a ROM, and a DVD (DVD-ROM and a DVD-R). 

As for the method of supplying the program, a 
client computer can be connected to a website on the 
Internet using a browser of the client, computer, and 
the computer program of the present invention or an 

10 automatically- installable compressed file of the 

program can be downloaded to a recording medium such 
as a hard disk. Further, the program of the present 
invention can be supplied by dividing the program code 
constituting the program into a plurality of files and 

15 downloading the files from different websites- In 
other words, a WWW (World Wide Web) server that 
downloads, to multiple users, the program files that 
implement the functions of the present invention by 
computer is also covered by the claims of the present 

20 invention. 

It is also possible to encrypt and store the 
program of the present invention on a storage medium 
such as a CD-ROM, distribute the storage medium to 
users, allow users who meet certain requirements to 

2 5 download decryption key information from a website via 
the Internet , and allow these users to decrypt the 
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encrypted program by using the key information, 
whereby the program is installed in the user computer. 

Besides the cases where the aforementioned 
functions according to the embodiments are implemented 
5 by executing the read program by computer, an 

operating system or the like running on the computer 
may perform all or a part of the actual processing so 
that the functions of the foregoing embodiments can be 
implemented by this processing. 
10 Furthermore, after the program read from the 

storage medium is written to a function expansion 

} 

board inserted into the computer or to a memory 
provided in a function expansion unit connected to the 
computer, a CPU or the like mounted on the function 

15 expansion board or function expansion unit performs 
all or a part of the actual processing so that the 
functions of the foregoing embodiments can be 
implemented by this processing. 

As many apparently widely different embodiments 

20 of the present invention can be made without departing 
from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the 
specific embodiments thereof except as defined in the 
appended claims . 

25 
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